Improved Color Reproduction by Hue Preservation in Integrated Multi-scale Retinex

Tone reproduction is now widely used in the field of HDR imaging and image enhancement, especially to provide the proper luminance, so that captured images give the same sensation as the real scene. As a result, a high contrast and naturalness of colors can be obtained. In recent studies on tone reproduction with the objective of reproducing natural looking colors in digital images, an integrated multi-scale retinex (IMSR) has produced great naturalness in the resulting images. Most methods, including IMSR, work in RGB or quasi-RGB color spaces, although some methods have adopted the use of luminance. As such, this produces hue distortion from the perspective of the human visual system, that is, hue distortion in CIELAB color space. Accordingly, this paper proposes an enhanced IMSR method in a deviceindependent color space, CIELAB, to preserve the hue and obtain a high contrast and naturalness. To achieve the desired objectives, a captured sRGB image is transformed into CIELAB color space. The IMSR is then applied to only the L values, thereby preserving the balance of the color components. However, since this process causes unnatural saturation, saturation adjustment is performed by applying the ratio of the chroma variation at the sRGB gamut boundary according to the corrected luminance. The adjusted CIELAB values are then transformed into sRGB using an inverse transform function. In experiments, the proposed method is shown to improve the visibility in dark shadows and bright regions in the resulting images and reduce any color distortion. Introduction Tone reproduction or tone mapping is achieved through an operator attempting to replicate the sensation of real-world luminance intensities in images from a regular digital camera. Most digital camera sensors are limited as regards capturing both the high and low luminance regions of a scene. In fact, such devices can only sense a dynamic range of about 10cd/m, whereas the dynamic range of a real scene is almost 10cd/m. Therefore, unless properly treated, images captured from regular digital cameras only present light or dark regions. Meanwhile, the dynamic range of the human eye is almost 10cd/m, which also does not cover the entire dynamic range of a scene, however, the human visual system uses a mechanism called “lightness adaptation” to perceive the light and dark areas of a scene simultaneously[7][8]. Thus, to obtain a similar perception when looking at digital images, tone reproduction methods are used. There are two types of tone reproduction technique: TRC (spatially-invariant tone reproduction curve) and TRO (spatiallyvariant tone reproduction operator)[3]. TRC methods are based on the global adaptation mechanism of human vision and performed pointwise on the image data. However, while simple and efficient, these methods do not preserve the local contrast in the case of both light and dark regions. In contrast, TRO methods are based on a multi-resolution decomposition algorithm, such as Gaussian decomposition, and operate in a local way on the image data (e.g. by convolution). As a result, the local image contrast is preserved, although unwanted spatial artifacts sometimes occur. Recently, various Retinex methods, which are TRO methods, have been widely used, due to their high local contrast and improved visibility, even given the presence of artifacts, such as halo effects. Jobson and others also developed the Retinex theory into the single-scale retinex (SSR) method and multi-scale retinex (MSR) method, as a combined form of the SSR method[5]. Initially, the MSR method had problems related to appropriate values for the parameters, chromatic unbalance, color distortion, noise, and graying out. Yet, a lot of work has been dedicated to improve these issues. A multi-scale retinex with color restoration (MSRCR) attempted to overcome the graying out phenomenon in large uniform areas of an image by adopting a color restoration function to control the saturation of the final rendition[6]. However, the output does not reproduce natural tones and colors when compared with the real scene. Meanwhile, an adaptive scalegain Retinex algorithm was introduced to prevent halo artifacts and suppress chromatic unbalance and noise by synthesizing both the original image and the image processed by MSR[2]. In a recent paper, an integrated multi-scale retinex (IMSR) algorithm was introduced to improve the visibility in dark shadow areas of natural color images, while preserving a pleasing contrast without banding artifacts[1]. In this case, a Gaussian pyramid decomposition is used to reduce the computation time for generating a large-scale surround, while an integrated luminance surround value is applied to each channel to preserve the color balance in RGB color space. Other papers have also attempted to prevent color or chroma distortion by controlling the ratio of the RGB channels. However, regardless of such efforts, all existing methods lead to some perceived color distortion. Based on the assumption that the input images are directly acquired in sRGB, the execution of MSR in RGB color space does not preserve the perceived hue, that is, the hue of the original image in CIELAB color space is distorted. Accordingly, this paper proposes an enhanced IMSR method in a device-independent color space, namely CIELAB, to preserve the hue. First, a captured sRGB image is transformed into CIEXYZ color space and then into CIELAB color space to calculate the lightness, hue, and saturation. The IMSR is then only applied to the lightness values, thereby preserving the balance of the color components. Thereafter, the L values produced by the IMSR are mapped to displayable values by means of a cumulative 248 ©2009 Society for Imaging Science and Technology distribution function to preserve the luminance in the high valued regions. However, since this process results in an unnatural saturation, a saturation adjustment according to the changed luminance is applied to the a and b channels. Based on an analysis of the chroma value variation with the IMSR, the ratio of the chroma variation is adjusted at the sRGB gamut boundary depending on the luminance variation to achieve color naturalness. Finally, the adjusted CIELAB values are transformed into sRGB using an inverse transform function. In experiments with real scene images, the results show a high visibility in both dark and bright regions, and natural colors without any hue distortion. Furthermore, in the case of an observer preference test, most observers perceived the images from the proposed method as more natural than previous results. Previous Integrated Multi-scale Retinex Method The IMSR method proposed by Wang[1] is based on the adaptive scale-gain retinex developed by Kotera et al[2] with certain differences. First, the IMSR adopts a linear space without a logarithmic conversion to avoid any uncertainty for noise and the output range spreading in dark shadows. Second, the IMSR only uses the luminance channel to form the surround image, and then applies this result to each color channel to maintain the color balance. As such, the main difference is the use of an integrated multi-scale luminance surround from multiple luminance surround images using Gaussian filters with a different standard deviation. The whole process is illustrated in Figure 1. Preserving the color balance is achieved by applying the integrated surround images Ssum to each channel in the IMSR. The following equation describes the IMSR process: ( , ) ( , , ) , ( , , ) i